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Introduction 

Developing a culture of evidence to assess and improve teacher preparation 
programs is a critical issue in American education. Teacher education has been 
struggling with the challenge of preparing and retaining sufficient numbers of 
high-quality teachers who can work effectively with students from all cultural and 
racial backgrounds, raisingtheachievementforall students (Wang, Spalding, Odell, 
Klecka, & Lin, 2010). Darling-Hammond (2002) found that teacher preparation 

is a stronger correlate of student achievement than 
class size or school spending, accounting for 40% to 
60% of the variance in achievement. Teachers who 
learn and practice sound pedagogical techniques can 
affect students' measured achievement (Blair, 2000). 
Although these studies indicate that teacher qual¬ 
ity is the most important factor influencing student 
achievement (Whitehurst, 2002), even among those 
who believethehigh quality preparation of teachersis 
critical, there are sharp contrasts concerning the best 
approach (Levine, 2006). 

M any scholars suggest that a strong research base 
on how bestto prepare teachers to meetthechallenges 
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of today's classrooms is lacking (Wilson, Floden, & Ferrini-M undy, 2001). M etzler 
and Blankenship (2008) discussed a "paucity of systematically collected evidence" 
i n teacher preparati on assessment despite it bei ng central to the conduct and future 
of teacher education (p. 1098). Cochran-Smith (2003) posited that formal program 
assessment efforts are noticeably lacking in teacher education. This shortage of 
evidence results in a myriad of potential "solutions" regarding teacher preparation, 
but with few ways to evaluate their promise (Boyd, Grossman, Lankford, Loeb, & 
Wyckoff, 2008). 

N umerous reports and analyses havefocused on this lack of a research base with 
most demanding better and more authentic assessment (Darling-Flammond, 2006). 
Concurrently there is a national demand for the reform of teacher education, particu¬ 
larly university-based preparation (Capraro, Capraro, & H elfeldt, 2010). Educational 
coursework has been found to have a critical point of diminishing returns and several 
studies have indicated that teachers with advanced subject matter degrees, rather 
than advanced education degrees, produce students who perform better in math and 
reading (Kaplan & Owings, 2002). A credential in education may be sufficient to 
produce student learning, but greater content knowledge has been found to affect 
learning as much as advanced education degrees (Greenwald & Fledges, 1996). 

Eleven years ago Zeichner (1999) pointed out that education faculty must do 
the best job possible in preparing teachers for our schools or perhaps let someone 
else do the job. M any voices echo that sentiment, including Secretary of Education 
Arne Duncan (2010) who asserted that many of the nation's 1,450 schools, colleges, 
and departments of education are doing a mediocre job of preparing teachers for 
the realities of the 21st century classroom. This type of change requires quality 
assessment and a clear understanding of what the resulting data indicate. 

Theevidence-based education movement, which holdsthat decisions about prac¬ 
tice and policy should be made on the basis of empirical evidence about outcomes, is 
now predominant (M oss, 2007), despite the defensiveness and recalcitrance of some 
faculties of education (Akmal & M iller, 2003). M any initi atives are intended to create 
new cultures of evidence or inquiry in institutions (Knapp, Copland, & Swinnerton, 
2007) and/or to "re-culture" organizations so that using evidence and assessment 
data becomes central to the way decisions about local policy and practice are made 
(Louis, 2008). Cochran-Smith (2009) calledfornew culturesof evidenceand inquiry 
in teacher education and stated that they have the potential to be transformative and 
revitalizing. She also pointed out that current discussions about creating culturesof 
evidence in teacher preparation often do not reflect the understanding of culture or 
its resistance to change. Gee (2007) stated that in assessment of teacher preparation 
there is a conspicuous absence of cultural nuance, including an absence of situated 
understandings of theroleof human interpretation in constituting and using evidence. 
While many reports discuss the need to rely on evidence in making programmatic 
decisions, there is little discussion about how such a system would coincide with the 
local cultures of colleges and universities (Cochran-Smith, 2009). 
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Cochran-Smith and Zeichner (2005) discussed the difficulty of the research 
needed to improve teacher preparation programs and pointed out that it depends on 
several critical I inks which could connectteacher preparation prog rams with candidate 
know I edge, ski 11 s, and di sposi ti ons, and the candi dates' actual practi ces i n cl assrooms, 
eventually linking to pupil learning while in the graduates' charge. They stated, 

... unraveling the complicated relationships between and among these variables 
and the contexts and conditions in which they occur is exceedingly complex, and of 
course this enti re enterprise assumes in the first place that there is consensus about 
appropriate and valid outcome measures, an assumption that is arguable, (p. 3) 

What Is Valid Evidence? 

TheTeachersfora New Era ini tiative proposed a conceptual framework for the 
development of an evidence portfolio to demonstrate and assess a teacher education 
program's success in preparing teachers (Cochran-Smith, 2009). While there are 
multiplecomponentsto such a portfolio, the first area of importance is a survey and 
tracking of graduates. Blanton, Sindelar, and Correa (2006) identified large-scale 
surveys, teacher checklists, and comparison to standards as three of the five ways 
in which beginning teacher preparation quality may be examined. There is general 
agreement that a teacher graduate's effect on student achievement is an important 
variable to examine a program’s effectiveness, yet no proven methodology exists 
for accomplishing this. 

The "value added” approach in particular is under attack (Baker etal., 2010). 
Teacher performance assessments using teacher work sample methodology de¬ 
veloped by The Renaissance Group are useful for examining individual teacher's 
effects on achievement (Torgerson, M acy, Beare, & Tanner, 2009) but have not 
been used systematically for program evaluation. Despite these evidence-oriented 
initiatives, little has been done to evaluate the quality of the evidence being gener¬ 
ated or devel op systemati c w ays to use that ev i dence to i m prove teac her preparati o n 
(Ludlow etal., 2008). 


Present Research 

Thisstudy investigated theeffect selected extrinsic variables haveon survey data 
collected to determine the efficacy of, and improve, teacher preparation programs. 
While recognizing other aspects of program evaluation, isolating the effects of ex¬ 
trinsic variables on the survey results is an important step to determining whether 
the results can be accepted atfacevalueorif they areinfluenced by outside factors 
over which programs have no control. 

In working toward a culture of evidence concerning teacher preparation, all 
schools, departments, and col leg esof education of the23 Cal iforni a State U niversity 
(CSU) campuses established common assessments as recommended by Cochran- 
Smith (2009) and Darling-Hammond (2006). In 1999, a survey of credentialed 
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graduates at the end of their first year of professional teaching and the graduate’s 
employment supervisor during that year of teaching was initiated by the CSU 
Education deans and the system Chancel lor. The survey contains specific questions 
about the quality of preparation provided by thecredential program. Each campus 
receives an annual report from the CSU Center forTeacher Quality (CTQ) with 
results from the survey concerning the previous year’s graduates and supervisors. 
The report also includes a summary of all data since the inception of the surveys 
for comparison purposes, and parallel results for the 23 CSU campuses compiled 
system-wide.Thisuniqueserviceallowseachcampusto track theeffectsof program 
changes designed to improve performance. 

As Ludlow etal. (2008) predicted, many campuses have struggled to develop 
systematic ways of using this rich body of evidence to improve teacher preparation. 
Teacher education faculty are well aware of the complex web of variables described 
by Cochran-Smith and Zeichner (2005). They often question the survey results, 
citing extrinsic factors to explain differences between the scores obtained by their 
own program versus those obtained elsewhere on campus or in the greater CSU 
(Beare, 2009). Beliefs concerning some factors are based on what "a priori" may 
seem important, (such as, the number of university credits required orthe number 
of students in a program) and some are based on conventional wisdom concerning 
important K-12 school characteristics (e.g., socio-economic status, English Lan¬ 
guage ability of students, or school achievement level). This study examined two 
factors specific to the preparation programs, and four extrinsic factors specific to 
the K-12 schools in which surveyed graduates were teaching. 


Survey Instrument 

The Systemwide Evaluation of Professional Teacher Preparation Programs 
(SEPTPP) compiles evidence about the extent to which K-12 teachers who are 
recent graduates of credential programs on CSU campuses are prepared for their 
most important teaching responsibilities and the extent to which CSU professional 
coursework and fieldwork were professionally valuable and helpful to them during 
their initial year of K-12 teaching (CTQ, 2009). This is accomplished by asking 
both graduates and the graduates' employment supervisors to complete separate, 
but parallel, 110-item online surveys at the end of the graduate's first year of full 
time professional teaching employment. 

The instrument includes common questions for all teachers and supervisors as 
well as credential-specific questions for particular groups. They are queried about 
the extent to which the teachers were prepared for important responsibilities that 
are commonly assigned to K -12 teachers. Teachers are also asked common ques¬ 
tions about the extent to which major features of the preparation programs proved 
to be valuable and helpful during subsequent teaching. Finally, respondents reply 
to questions about the quality of the credential programs in relation to prominent 
standardsforstateand national accreditation.Teachersandsupervisorsarealso asked 
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credential-specific questions about the extent to which the teachers were prepared 
for responsibilities that are most commonly associated with their specific assignment 
(CTQ, 2009). Supervisors are asked to evaluate new teachers as novices, and only if 
they have observed and had a conference with them during this first year. 

Data Collection 

Each CSU campus forwards to the CTQ a list of former students at that campus 
who, during a prior 12-month period, met the standards for state certification as 
K-12 teachers. School sites are identified for approximately 55% of the completers 
from these sources. After receiving an initial list, the CTQ and CSU campuses 
make a second effort to find the school locations of additional teachers by directly 
contacting approximately 1,000 school districts and 50 county offices of educa¬ 
tion. Thiseffort yields site information for an additional 30 percent of recentCSU 
completers (CTQ, 2009). 


Validity of the Evaluation 

The validity of the evaluation derives from the alignment between the evalua¬ 
tion questions and (1) C alifornia standards for grades K-12 in all curriculum areas, 
(2) California Standards for Accreditation of Professional Teacher Preparation, (3) 
California Teaching Performance Expectations, (4) California Standards for the 
Teaching Profession, and (5) Standards adopted for institutional accreditation by 
the National Council for Accreditation of Teacher Education (CTQ, 2009). Indi¬ 
viduals who had participated in drafting and implementing California’s accredita¬ 
tion standards for universities and its performance expectations for teachers were 
responsible for the alignment of the evaluation questions (CTQ, 2006). 

Reliability of the Evaluation 

U ncertainty about evaluation findings comes from two principal sources, the 
number of evaluation participants and the extent of their concurrence with each 
other. The evaluation findings become increasingly certain to the extent that the 
questions are answered by increasing numbers of program completers and their 
employment supervisors. Each year the data set yields the percent of respondents 
who gave specified answers to each item and includes reliability estimates in the 
form of confidence intervals based on the number of respondents and the concur¬ 
rence or homogeneity of responses, in 2003, theCSU Deans of Education grouped 
together substantively related evaluation questions into "composites.” For example, 
the survey i ncl udes several questions about preparing teachers for diversity i n edu¬ 
cation. The deans grouped these questions together in a composite called Prepar¬ 
ing for Equity and Diversity in Education. These groupings facilitate the analysis 
and interpretation of large amounts of complex data and the composite scores are 
substantially more reliable than are the participants' responses to individual survey 
questions and are sufficiently valid and reliable to serve as the basis for academic 
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and professional decisions about teacher preparation (CTQ, 2006). The reliability 
for the composite scores for the system and the individual campuses generally 
range from 0 to 2 percentage points at the 90% confidence level. 

Research Question 

D o specific extrinsic variablessignificantlyinfluenceresultsofsurveysevaluating 
CSU teacher preparation programs? This concern stated by Ludlow et al. (2008) 
has been omnipresent in discussions of the CSU survey data by faculty and deans. 
Both groups question if extrinsic variables are responsible for survey outcomes 
instead of or in addition to the actual preparation program strengths or weaknesses 
in specific assessment areas (Beare, 2009). To address this question, the present 
research examined the effect of specific extrinsic variables on the preparation pro¬ 
gram ratings by supervisors and graduates across the 23 campus CSU system. 

Operational Definitions of Extrinsic (Independent) Variables Examined: 

Credential Program Variables 

Credit requirement: The total number of semester units (credits) required to 
completea credential program, i n theC SU this ranged from 32 to 56, as represented 
in university catalogs. 

Program size: The number of candidates who completed a CSU credential 
program in oneyear. For the year of study, theCSU credential ed 6,667 elementary 
teachers, with a mean for campuses of 303 and a range from 66 to 570 teachers 
(CCTC, 2008). 


K-12 School Variables 

Socioeconomicstatusof students:T he percentageof students i n each graduate's 
class who qualify for free or reduced lunch. 

Language status of students: The percentage of students in the graduate's 
class who are classified as English Language Learners. 

Achievement level of the school: The decile ranking of the graduate's school of 
employment on theCalifornia Standardized Testing and Reporting (STAR) results. 

Preparation of other teachers at the graduate's school: The percent of teach¬ 
ers in each school who were teaching on an "emergency permit” only. 


Two Steps 

This research was carried out in two steps. First, the effects of credit require¬ 
ments and program size were examined by analyzing selected composite scores for 
respondent graduatesand their employment supervisors on theSEP TP P. Second, the 
effect of the four student-related variables on selected composite SEPTPP scores 
for supervisors were examined. 
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Methods and Results 

Study I: Effect of Credential Program Variables on Composite Scores 

Subjects: The subjects for the investigation of the effect of the two program 
variables on composite scores were graduates from a C SU elementary teacher cre¬ 
dential program after oneyear of teaching (N =994) and their employmentsupervisors 
(N =242). Over 99% of the supervisors were school site principals (CTQ, 2009). 
According to survey results, 93% of employment supervisors reported visiting the 
first year teacher six or more times, and over 50% reported having greater than 
six conferences about teaching with the graduate. Graduates reported differently, 
with 78% reporting five or fewer observations by the principal, and 84% reporting 
five or fewer conferences about teaching with the principal. Either set of responses 
reveals multi pie opportunities for supervisors to becomefami liar with the teacher's 
performance. 

Dependent variables: The composites selected to examine the effect of the two 
credential programvariablesonratingsby principalsrepresentthefollowing important 
responsibilities of K-8 teachers: "How well prepared was the graduate to..." 

• know and understand the subjects of the curriculum at the K-8 grade level? 

• plan instruction and prepare classroom materials and activities for instruction? 

• use an appropriate mix of effective teaching strategies in the classroom? 

• meet the instructional needs of English Language Learners? 

• understand child development, human learning, and the purposes of school? 

• teach reading-language arts (K-8) according to state standards for the grade assigned? 

•teach mathematics (K-8) according to state standards for the grade assigned? 

•teach visual and performing arts according to state standards for the grade assigned? 

The selected composites for the graduates represent their preparation to teach 
the folIowing specific content areas: "How well prepared were you to..." 

•teach reading-language arts (K-8) according to state standards for your grade? 

•teach mathematics (K-8) according to state standards for the your grade? 

• teach visual and performing arts according to state standards for your grade? 

Correlation coefficients: Two-tailed Pearson correlations wereused to investi¬ 
gate therelationship between thedependentvariablesand the independent credential 
program variables. Table 1 shows the correlation of principals' evaluation of the 
graduates' preparation with both the number of units required in each program 
and the number of credentials awarded for the year. Results showed no significant 
correlations between the independent and dependent variables. The correlations 
were small, ranging from -.066 to .062 for number of units and -.098 to .028 for 
the number candidates completing the program. As may be incidentally seen, 
the correlations among the dependent variables are all statistically and clinically 
significant. For example, the correlation between preparation to teach math and 
preparation to teach reading language arts was .942. 
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Table 2 shows the correlation of the graduates' composite ratings with the two 
independent credential program variables. A s with the principal evaluations, there 
were no significant correlations. The correlations ranged from .008 to .068 for 
program units and .019 to .046 for program size. Again, the relationship between 
the dependent variables were all statistically and clinically significant with the 
highest being .638, again between preparation to teach reading and preparation to 
teach mathematics. 

Because the lack of a relationship between program length and preparation 
rati ngsseemed counter-i ntuitive, a fol I ow-up examination of thedata was conducted 
in which the CSU programs were segmented into two groups, those with 44 or 
fewer units required and those with 45 or more units. The F-ratios did not reach 
the statistically significant level forany of thedependentvariables.Thesecomputa- 
tions suggest that during the first year of teaching the reported levels of readiness 
by program completers to perform important responsibilities of K-8 teachers are 
not substantively related to the relative length of their credential preparation. 


Table I 

Pearson Correlations of Principal Evaluation ofTeacher’s Preparation 
with the Number of Semester Hours Required in theTeacher’s 
Credential Program, the Number of Credentials Issued, 
and the Inter-correlations Among the Evaluation Items (N=242) 


Evaluation 

Item 1 2 3 4 5 6 7 8 

# Sem. 
Hours 

# of 
Cred. 

Effectiveness 
of Preparation to: 

1. Understand 

Curriculum .776* .673* .621 * .703* .769* .745* ,576* 

-.013 

-.091 

2. Plan Instruct & 

Class Activities .776* .755* .683* .697* .841* .819* .496* 

-.066 

-.039 

3. Manage Class 

for Instruction .673* .755* .622* .639* .716* .688* .453* 

-.058 

.024 

4. Meet the Needs 

of ELL Students .621 * .683* .622* .734* .71 1 * .676* ,597* 

-.045 

-.044 

5. Understand 

Growth & Develop. .703* .697* .639* .734* .699* .685* .641 * 

.062 

-.098 

6. Teach Reading/ 

Language Arts (K-8) .769* .841 * .716* .71 1 * .699* .942* .482* 

-.012 

-.042 

7. Teach 

Mathematics (K-8) .745* .819* .688* .676* .685* .942* .455* 

-.06 

-.031 

8. "leach Visual- 

Perform Arts (K-8) .576* .496* .453* .597* .641 * .482* .455* _ 

-.067 

.028 

* pc.OI, two-tailed. 
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Study 2: Effect of K-12 School Characteristics on Composite Ratings 

Subjects: The subjects for the investigation of the effect of the K-12 school 
variables on composite ratings were the employment supervisors of all elementary 
program completers since the initiation of SEPTPP who provided information 
for the four independent variables. A total of 19,050 supervisors responded to the 
survey overa 10-year period with responses for each independentvariable ranging 
from 12,847 to 18,287. 

Dependentvariables: The composites selected to examine theeffecton employ¬ 
ment supervisors ratings of the characteristics of the schools in which the graduates 
were employed during their first year of teaching represent the following important 
responsibilities of K-8 teachers: "How well prepared was the graduate to..." 

• meet the instructional needs of ELL learners? 

• meet the instructional needs of learners from diverse backgrounds? 

• meet the instructional needs of students with special learning needs? 

• know about resources in school and community for students and families at-risk? 

• communicate with parents or guardians of his/her students? 

•teach reading language arts according to the CA Content Standards in Reading? 

•teach math according to the CA Content Standards in Mathematics? 

•use language so pupils at different levels of understand oral and written English? 

Correlation coefficients: Table 3 shows the correlation between the four 
independent K-12 school variables and each of the eight dependent variables as 
well as the inter-correlations between the four independent factors. Each of the 
independent variables had a statistically significant correlation with at least one 
aspectof teacher preparation.Thedata, however, show that none of the correlations 


Table 2 

Pearson Correlations of Graduates’ Evaluation ofTheirTeacher 
Preparation with the Number of Units in Their Credential Program, 
the Number of Credentials Issued and the Intercorrelations Among 
the Evaluation Items (N = 994) 


Evaluation Item 1 

2 

3 

Semester 

Hours 

Required 

Number of 
Credentials 
Issued 

Effectiveness of Preparation to: 

1. Teach Reading 

Language Arts (K-8) 

.638* 

.520* 

.047 

.019 

2. Teach Math (K-8) .638* 


.488* 

.008 

.020 

3. Teach Visual 

Performing Arts (K-8) .520* 

.488* 


.068 

.046 


* p<.01, two-tailed. 
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Table 3 

Correlation of CSU System-wide Principal Evaluation ofTeachers’ 
Preparation with Four Demographic Characteristics of the School 


How well prepared % of Students 

% of Students 

API Decile 

% of Teachers 

was this teacher to Eligible for 

Who Are English 

of School 

in School with 

begin each aspect Free or Reduced 

Language 

Last Year 

Emergency 

of a teacher’s job? Lunch 

Learners 


Permit 

Number of 

Respondents 12,847 

18,287 

17,325 

16,701 

Meeting needs 

of ELL .009 

.004 

.015 

.024* 

Meeting needs of 
diverse students .009 

-.001 

.021 

.017* 

Meeting needs of 
students with special 
learning needs .04* 

Knowing about 
resources for 

.003 

.06* 

.02* 

at-risk students .02* 

.004 

.019* 

.017* 

Communicating 
with parents 

or guardians .010 

"leaching standards- 

-.003 

.04* 

.007 

based reading/ 
language arts .045* 

"leaching 

standards-based 

-.038* 

.096* 

.015 

mathematics .036 

Use language 
so all pupils 
understand oral 

-.042* 

.093* 

.01 1 

and written English? .035 

-.001 

.062* 

.018 

Intercorrelations 

% Eligible Free/ 

Reduced Lunch 

.412* 

.408* 

.047* 

% Students who 

are ELL .412* 

API Decile of 

— 

-.035* 

-.028* 

School Last Year .408* 

% of Emergency 

Credentialed 

-.035* 

— 

.003 

Teachers .047* 

-.028* 

.003 



Note. A total of 19,050 Supervisors responded to the survey. Different numbers of supervisors answered 
each of the questions thus the varying number of respondents. 

* p < .05, two-tailed. 
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reached the .10 level. A correlation between two sets of scores reflects whether 
there is a consistent, predictable association between the scores. Researchers may 
interpret the magnitude and direction of the correlations as they stand, though most 
researchers prefer to square the correlation and use the resulting value to measure 
the strength of the relationship (C reswell, 2005).The coefficients of determination 
show that less than 1% of the variance of the supervisors’ ratings is explained by 
any of the i ndependent vari abl es. 

An examination of Table 3 shows that, as might be expected, a relationship 
between theSES at the schools, as measured by the percent eligible for free lunch, 
is strongly related to both the percentage of students who are ELL (r=.412) and 
the achievement level of the school (r=.408). The stereotype that low achieving 
schools are staffed with emergency permitted teachers was not demonstrated by 
this data. The correlations with the other independent variables were all less than 
.05, showing no clinical significance. 

A s was the case w i th the credenti al program vari abl es, these computati ons taken 
together suggest that duri ng the fi rst year of teachi ng the reported I evel s of readi ness 
by program completers to perform important responsibilities of teachers were not 
substantively related to conditions in theschoolsthat are generally considered among 
educators, legislators, and the media and public to bean extreme challenge. 


Discussion and Conclusions 

Survey data is an important source of information for program assessment 
(B lanton etal., 2006; Cochran-Smith, 2009; Dari ing-Hammond, 2006). W hi lestudent 
achievement data, process-product measures, and comparison to standards are also 
essential components of a comprehensive system to evaluate teacher preparation 
programs, the present research specifically examined the effect of certain extrin¬ 
sic variables on principal and graduates' assessment of the graduates' university 
preparation program. 

The impetus for such an examination lay with the lack of a culture of evidence 
at the campuses that have utilized the SEPTPP data. Cochran-Smith (2009) warned 
of a possi ble col I ision between the local culture of universiti es and the evi dence used 
to examinefor program quality. Gee(2007) and Phillips (2007) foreshadowed it, and 
Cochran-Smith and Zeichner (2005) overtly pointed out that there will be arguments 
aboutany program assessment data presented becauseof thecomplex web of variables 
involved in linking outcome measures to university program features. 

The common experience across the CSU has been that, when faced with data 
reflecting less positively on the preparation program than they would like, faculty 
counter it with the rationale that the data reflect the external variables that were 
exami ned here. Study 1 addressed the most frequent argument, that more course- 
work would improve assessment data. California does notallow an undergraduate 
major in education so credentials are added on to degrees in other subjects and 
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programs are also limited to preparation that can be completed in one year. Also 
stated frequently by faculty has been the belief that large preparation programs make 
students feel "distant" from instructors or professionally isolated. Neither the number 
of units required by the various programs nor the number of candidates completing a 
credential in a given year had any discernable effect on assessment data. The minis¬ 
cule correlations between program length and the judged quality of preparation was 
surprising if not shocking. This data would seem to support the "point of diminishing 
returns” argument posited by Kaplan and Owings (2002) and should be seriously 
considered by all teacher education programs.The lack of effectfrom program size 
is less surprising, though it does counter those who advocate for small schools and 
small programs as a way to personalize and improve teacher preparation. 

The relatively strong inter-correlations among the various aspects of teacher 
preparation that were rated indicates that teachers judged strongly prepared in one 
area are so judged in other areas. This may mean that the factors that make one a 
wel l-prepared teacher general ize across al I areas of teachi ng or that strong prepara- 
tion programs produce teachers that are strong across the board. 

Study 2 addressed the assertion that teachers working in the most challenged 
schools, those with low SES, a high rate of ELLs, low achievement, and large 
numbers of emergency permitted teachers, will be judged less well prepared be¬ 
cause they are teaching under more challenging conditions. It is acknowledged that 
new teachers are often placed in these schools because teachers with seniority flee 
these conditions when possible (Byrd-Blake et al., 2010). The results of Study 2, 
however, showed no clinically significant correlation between the principal s' evalu¬ 
ation of theCSU graduates’ preparation program and the characteristics of schools 
in which they taught during their first year. None of the variables reached even a 
minimal level of relationship. It is thus clear that principals’ judgment concerning 
the quality of a teacher's preparation was not affected by the school characteristics 
that are typically thought of as indicating difficult teaching conditions. 

Thefollow-up survey of university-based teacher preparation prog ram graduates 
and employmentsupervisorsconducted by theCSU isunprecedented.Aspredictedby 
the literature, however, some involved have been reluctantto acceptthis opportunity 
to utilize the culture of evidence so increasingly necessary in the field of teacher 
preparation.This research contributes knowledgeto this critical area by addressing the 
extent to which the results of surveys assessing university-based teacher preparation 
are influenced by extrinsic variables over which a program has little or no control. 
The lack of relevant correlation found by these studies indicates that survey results 
can and should be used by programs to strengthen their preparation of future teachers 
without significant worry of contamination from the extrinsic variables examined. 

Next Steps 

Results of this study provide clear indications for future research to validate 
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the culture of evidence for program improvement in teacher education. Individual 
characteristics of the teachers being evaluated and effect of teacher ethnicity and 
background on supervisor evaluation should be studied. In addition, various pathways 
to becomi ng a teacher, i ncl udi ng traditi onal campus-based preparation, professi onal 
development schools, residency programs, online programs, and programs where 
candidates are employed as teachers while completing their credential, should be 
compared both system-wide and within individual institutions. 

A most important step in the extension of this study will be the triangulation 
of data including SEPTPP ratings, teacher performance assessments, and student 
achievement. While the statistical method known as "value added” is still being 
strongly questioned asa high stakes method (Bakeret. al., 2010), student achievement 
is a factor that should be considered part of a rating of program efficacy. As stated, 
teacherwork samplesand performanceassessments are useful forexamining learning 
taking placein a classroom. Specific school achievement levels, by grade, subject, and 
subgroup is easily obtainable for all schools in California through the Educational 
Results Partnership website. While it may not be advised to use this data to evaluate 
individual teachers, itisfacilitativeto examinetheeffectsa university-based program 
has on achievement at professi onal development schools or the locales where student 
teachers are placed. Finally, a comparison of traditional university-based teacher 
preparation with alternative programs such as Teach for America, paraprofessional 
teacher preparation programs, and for-profit institutions would provide valuable 
information for program improvement and would inform policymakers and future 
teachers about the validity of such alternatve paths to teaching. 
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